The demand for digital video products continues to increase. Some examples of applications for digital video include video communication, security and surveillance, industrial automation, and entertainment (e.g., DV, HDTV, satellite TV, set-top boxes, Internet video streaming, digital cameras, cellular telephones, video jukeboxes, high-end displays and personal video recorders). Further, video applications are becoming increasingly mobile as a result of higher computation power in handsets, advances in battery technology, and high-speed wireless connectivity.
Video compression is an essential enabler for digital video products. Compression-decompression (CODEC) algorithms enable storage and transmission of digital video. In general, the encoding process of video compression generates coded representations of frames or subsets of frames. The encoded video bitstream, i.e., encoded video sequence, may include three types of frames: intracoded frames (I-frames), predictive coded frames (P-frames), and bi-directionally coded frames (B-frames). I-frames are coded without reference to other frames. P-frames are coded using motion compensated prediction from I-frames or P-frames. B-frames are coded using motion compensated prediction from both past and future reference frames. For encoding, all frames are divided into smaller blocks, e.g., 16×16 macroblocks of pixels in the luminance space and 8×8 macroblocks of pixels in the chrominance space for the simplest sub-sampling format.
Video coding standards (e.g., MPEG, H.264, etc.) are based on the hybrid video coding technique of block motion compensation and transform coding. Block motion compensation is used to remove temporal redundancy between adjacent frames in a video sequence and transform coding is used to remove spatial redundancy in the video sequence. Traditional block motion compensation schemes basically assume that objects in a scene undergo a displacement in the x- and y-directions from one frame to the next. Motion vectors are signaled from the encoder to the decoder to describe this motion. The decoder then uses the motion vectors to predict current frame data from previous reference frames.
This simple interframe coding technique does not consistently provide sufficient coding efficiency for many applications when there is temporal brightness variation across sequences of frames. Some video standards (e.g., H.264/AVC) provide for both implicit and explicit weighted prediction to help compensate for temporal brightness variation, which may provide enhanced coding efficiency for sequences with fades, lighting changes, etc. More specifically, weighted prediction (WP) provides for modification of motion-compensated prediction signals using weighting factors, e.g., a global multiplier and a global offset. In explicit weighted prediction, applicable to both P-frames and B-frames, the weighting factors are computed in the encoder and communicated to the decoder in the coded video sequence. In implicit weighted prediction, which is typically applicable only to B-frames, the weighting factors are automatically computed during encoding based on the temporal distance between a current frame and two reference frames and need not be communicated to the decoder.
Further, in some weighted prediction techniques, each 8×8 block in P frame can have multiple candidate reference frames and the best reference frame is selected from the candidates. For a B frame, a block can have one or two (forward and backward) reference frames. When a single reference frame is used, the weighted prediction image=weight*prediction image+offset. When two reference frames are used, the weighted prediction image=average of forward weighted prediction image and backward weighted prediction image, where the forward weighted prediction image=forward weight*forward prediction image+forward offset and the backward weighted prediction image=backward weight*backward prediction image+backward offset.
The use of weighted prediction has been shown to provide coding gains for sequences of frames with global brightness changes. However, it has also been shown to result in coding losses for sequences of frames without global brightness changes.